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INTERVENTION/PREVENTION PROGRAM EVALUATION 

1998-1999 

In 1994-95, the Wake County Public School System received a grant from the North Carolina 
Department of Public Instruction to implement a pilot project known as the 
Intervention/Prevention Program (I/P). The main purpose was to provide better support to 
students who were at-risk by collaborating with many human service agencies that work with 
children and families. The vehicle for change was the development of Student Support Teams 
(SSTs) to support students as a replacement for existing “teacher assistance teams”. For the 
1994-95 school year, a district/community committee selected nine pilot schools through a 
competitive process for program participation. 

The SST is a collaborative, multidisciplinary team that is responsible for developing action plans 
that support and strengthen individual students through the use of school and/or community- 
based strategies. The SST model involves viewing students from a holistic perspective, that is, 
the SST must be inquisitive about the dynamics of all aspects in the student’s environment and 
life circumstances to design appropriate and effective ways to increase success (e.g., academics, 
family, friends, affect, culture, behavior). During the 1994-95 school year the Coordinating 
Committee of the I/P program wrote and developed the Guide to Strong SSTs. Guide to Strong 
SSTs provided the framework for developing and implementing a successful SST process at an 
elementary, middle, or high school by defining specific goals for students and developing a plan 
of action to meet those goals. 

Since 1994, the North Carolina Department of Public Instruction has continued to provide grant 
funding that supports the Intervention/Prevention Program in the Wake County Public School 
System. The program's mission is to provide support to students and youth who are experiencing 
academic, behavioral, family, and emotional difficulties which interfere with learning. The SST 
at each school provided this support through a collaborative team approach, generating 
individualized strategies that influence the interaction of environment, family, emotions, and 
learning on the child’s life. The project grew from the original group of nine schools piloting the 
project in 1994 (six elementary schools and three alternative schools), to 105 schools (69 
elementary, 20 middle, 13 high, and 3 alternative school sites) during the 1998-99 school year. 

The Evaluation and Research Department has collected and analyzed data in order to make 
summative evaluations of the impact of the SST interventions and services on the students and 
youth served. A previous SST report, Intervention/Prevention Grant Evaluation 1996-9% (Report 
No. 99.16), focused on the 1996-97 and 1997-98 school years. This report involved a systematic 
process of data collection and generation to evaluate student performance, areas of concern, and 
strategy effectiveness. The current report focuses on the 1998-99 school year. As a result of 
experiences from the initial report, there were some minor changes in the data collected. . The 
data discussed in this report come from the final reports completed by the SST chairpersons for 
each of the students who were discussed at the SST meetings in 1998-99. 
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PROGRAM DESCRIPTION 



Student Support Teams are multidisciplinary (multi-agency) school-based teams that design and 
implement individualized action plans to strengthen and support students through school-based 
and/or community-based strategies. Implementation of the SST model consists of extensive 
training in three guiding practices. 

Individualized Interventions-, generating strategies that influence the interaction of holistic 
influences on the child’s life (e.g., environment, family, emotional, learning). 

Collaborative Team Approach-, involving parents, school staff, and community agency 
representatives in partnership to provide services needed by the child. 

Capitalizing on Strengths-, remediating student weaknesses by building on strengths. 

The SST approach is designed to assist schools in helping students-at-risk. Many of these 
students are challenged with a variety of concerns at school and at home. These challenges 
include health, education, and social service needs, which may negatively affect the learning 
process. 

In public schools, a variety of factors associated with school failure may affect students at risk. 
Examples of risk factor conditions external to the school include poverty, unmet health needs 
and physical, emotional, and/or sexual abuse. Neither a single school nor a human service 
agency alone is capable of addressing the plethora of challenges these students face. In addition, 
students and families may often have difficulty in accessing school and community services. As 
a consequence, students and families with numerous concerns may not know where or how to 
seek assistance. 

The school based SST builds and renews connections and partnerships among school and 
community based human service agencies to provide help with risk factors. Several essential 
functions portray SSTs at the school site. These include, assessment, development of an action 
plan, linking students to services, implementation, monitoring, and evaluation. Through 
collaborative interactions, SSTs establish a comprehensive service network where participants 
share the responsibility for students at risk of school failure and their families. Therefore, 
students and families receive a variety of services in a supportive, orderly, and coordinated 
fashion. 
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EVALUATION PLAN 

Major issues addressed in the 1996-97 & 1997-98 evaluation focused on the impact of the SST 
process and the identification of interventions that appeared to be more successful than others in 
improving presenting concerns. This focus was continued in the 1998-99 school year. The 
evaluation design also addressed specific questions about the SST program. 

Evaluation Questions 

The purpose of the 1998-99 evaluation was to respond to several questions related to the types of 
student referrals, the types of strategies recommended and implemented, and the effectiveness of 
the SST program. 

□ Were presenting concerns more prevalent in any particular area(s)? 

□ Did students who were referred for behavior concerns show improvement after SST 
interventions? 

□ Did students who were referred for emotional stability concerns show improvement after 
SST interventions? 

□ Did students who were referred for “other” concerns show improvement after SST 
interventions? 

□ Were strategies implemented when recommended for use? 

□ Were some strategies more effective than others? 

Data Collection Activities 

The Wake County School System's Intervention/Prevention staff collected data for the 1998-99 
school year from chairpersons for each school SST. Data were analyzed by an independent 
evaluator working for the school system’s Evaluation and Research Dept. Activities focused on 
measuring the impact of the SST interventions and services on the students served. A systematic 
process of data collection generated data to evaluate areas of concern and strategy effectiveness. 
Information in this report comes from the final reports completed by the SST chairpersons for 
each of the students who were discussed at the SST meetings in 1998-99. 

SST coordinators completed scan forms on each of the students referred to the team. The form 
collected data on the background of the student, who requested the intervention, the area(s) of 
concern, and the effectiveness of the individual strategies. Educational interventions 
encompassed areas of classroom-based strategies, school social work services, psychological 
services, family support services, school counseling services. Department of Health Services, 
community-based services referrals, and special education referrals. For 1998-99, strategies were 
grouped into three major classifications: classroom-based, school-based, and community-based. 
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SST coordinators were expected to collect the forms and verify that the information was correct. 
The data forms were submitted to the Department of Evaluation and Research for processing and 
data analysis. 



EVALUATION RESULTS 



School Participation in the SST Program 

Of the 105 schools participating in the SST program in WCPSS in 1998-99, there were 69 
elementary schools, 20 middle schools, 13 high schools, and 3 alternative schools. SST 
chairpersons completed scan forms for 3,586 students served during the school year. Figure 1 
summarizes the number of students who were reported as served. 

Figure 1 

Schools and Students Served By SSTs in 1998-99. 



School Type 


No. 

Schools 


Students 

Referred 


Elementary 


69 


2,632 


Middle ' 


20 


548 


High 


13 


352 


Alternative 


3 


53 


Missing Data 




1 


Total 


103 


3,586 



Previous evaluation reports identified 1,279 students served by SST in 49 schools in 1996-97, 
and 3,963 students served in 103 schools in 1997-98. 

At the end of 1998-99, there were a total of 90,953 students in the 105 schools that participated 
in the SST program. A total of 3,586 of these students, or four percent, were referred to SSTs. 
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Ethnic and Gender Characteristics of Referrals 

Figure 2 describes the ethnic and gender characteristics of the 1996-97 to 1998-99 students 
referred to the SSTs. In 1998-99, of those referred, approximately 35% (1241) were female and 
59% (2105) were male. For approximately 6% (240) of the students referred, the data on gender 
was unavailable. A total of 2093 males and 1238 females had information on their ethnicity, and 
information for 255 students was missing both ethnicity and gender information. 

Figure 2 

Characteristics of SST Referrals for 1996-97 to 1998-99. 





Black 


White 


Other 


1996-97 
Male (n = 831) 


41% 


55% 


4% 


Female (n = 448) 


48% 


50% 


2% 


Total (n = 1279) 


43% 


53% 


3% 


1997-98 


Male (n = 1960) 


42% 


53% 


5% 


Female (n = 1301) 


46% 


48% 


5% 


Total (n = 3261) 


44% 


51% 


5% 


1998-99 


Male (n = 2093) 


42% 


51% 


7% 


Female (n = 1238) 


45% 


47% 


8% 


Total (n = 3331) 


43% 


49% 


8% 



Over the last three school years the percentage of males referred to the SST teams has generally 
been steady at 60 to 65% of the referrals. The percentage of referrals that are Black has also 
remained fairly stable, but we have seen an increase in the percentage of students from other 
ethnic groups being referred. The proportion of the SST referrals that are White students has 
dropped from 53% in 1996-97 to 49% in 1998-99, with both White male and female students 
making up a smaller percentage of the referral group in 1998-99 than they did in both 1996-97 
and 1997-98. There is also a slight decline in the proportion of Black female SST referrals over 
the last three years. 
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Original Source of SST Referrals 



At the time of the initial referral, data was collected on the referral source for each student. 
Referrals could be initiated in a number of ways, including school staff, outside agencies, or 
parents. Figure 3 displays original referral sources for the school years 1996-97 to 1998-99 with 
percentages rounded. 

Figure 3 

Original Sources of SST Referrals, 1996-97 to 1998-99. 



Source of Request 


1996-97 


1997-98 


1998-99 


Administration 


2% 


2% 


2% 


Central Office 


N/A 


0% 


0% 


Guidance Counselor 


12% 


3% 


4% 


IRT / LART 


N/A 


0% 


0% 


Nurse 


1% 


0% 


0% 


Outside Agency 


1% 


0% 


0% 


Parent / Guardian 


18% 


14% 


17% 


Psychologist 


0% 


0% 


0% 


Teacher 


58% 


75% 


69% 


SAP Coordinator 


N/A 


0% 


1% 


Self-referral 


N/A 


N/A 


1% 


School Social Worker 


2% 


0% 


1% 


Special Education Teacher 


3% 


3% 


2% 


Other 


2% 


1% 


1% 



By a wide margin, regular education teachers (58% in 96-97, 75% in 97-98, and 69% in 98-99) 
referred students to the SST more than any other group, followed by parents (18%, 14%, 17%). 
During the 1996-97 school year, guidance counselors made 12% of the referrals while in the 
1997-98 school year, they made three percent of the referrals. This increased slightly to four 
percent in 1998-99. 

In 1998-99, a total of 2,352 students (66%) were reported as receiving pre-referral screening, and 
2,203 (61%) of the student’s parent/guardians were reported as having attended at least one of 
the SST meetings. 
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Categories of Presenting Concerns for SST Students 

For the 1998-99 school year, presenting areas of concern were divided into five major categories: 
(1) academic concerns, (2) behavior concerns, (3) psychological/emotional behaviors, (4) f amil y 
characteristics, and (5) other. The categories are summarized in Figure 4. 

Figure 4 

Categories of Pres^ting Concerns for the 1998-99 School Year. 



Academic 

Concerns 


Behavior 

Concerns 


Psychological / 
Emotional 
Behaviors 


Family Characteristics 


Other 


Handwriting 


Attention 


Lack of 
Confidence 


Lack of Basic Needs 


Attendance 


Math 

Computation 


Aggression 


Inexplicable 

Crying 


Death in a Family 


Hearing 


Math Problem 
Solving 


Assignment 

Completion 


Easily Frustrated 


Extended / Terminal 
Illness 


Medical 


Memory 


Following 

Directions 


Exhibits Grief 


Extended mental illness 
of immediate family 
member 


Motor Skills 


Oral 

Communication 


Organizational 

Skills 


Mood Changes 


Family changes (i.e., 
separation, divorce, 
new member, recent 
move) 


Retention 


Reading 


Social Skills 


Lack of 
Motivation 


Unemployment 


Speech / 
Language 


Reading 

Comprehension 


Social Skills 


Relationship 

Concerns 


Other 


Vision 


Written 

Language 


Withdrawn 


Low Self-esteem 
/ Seems 
Preoccupied 




Other 


Other 


Other 


Other 







all 



SST Summary Report - ver3.doc/cnd 



Presenting Concerns for SST Students 

For each student referred, major areas of concern were bubbled on the scannable form. More 
than one concern could be recorded for each student. The categories for major areas of concern 
were academic, behavior, family, psychological, and other. Of the 3,586 students that had final 
reports completed by the SST chairpersons, 248 students (7%) did not have any areas of 
concerns listed. Figure 5 summarizes the percentage of students in each of the five major areas of 
concern for the three school years. 



Figure 5 

Presenting Concerns in 1996-97 and 1998-99. 




Academic Behavior Family Psychological Other 

Presenting Concerns in Grades K-12 



For the 1998-99 school year, academic concerns were the most prevalent (77%). The prevalence 
of academic concerns reflects similar data from 1996-97 and 1997-98. In these years, 77 percent 
and 84 percent of students were referred to SSTs for academic concerns, compared to 77 percent 
in 1998-99. 

For all three school years, the second most prevalent concern was behavior (51% for 1996-97, 
81% for 1997-98 and 58% in 1998-99). The third most prevalent area of concern was 
psychological (45% for 1996-97, 65% for 1997-98, 51% for 1998-99). Family concerns were 
reported at 37% in both 1996-97 and 1997-98, and at 26% in 1998-99. Initial concerns coded in 
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the Other category (which can include attendance, hearing, medical, motor skills, retention, 
speech/language, vision, and other) has steadily decreased as a proportion of the referrals made 
to SSTs, from 39% in 1996-97 to 28% in 1998-99. 

Nineteen percent of the students had only one presenting area of concern, 24% had two areas of 
concern listed, 25% had three, 17% had four, and 8% (269 students) had all five areas of concern 
listed. 

Strategies Recommended for Presenting Concerns 

For each student, the SST team recommended potential strategies. Multiple strategies could be 
recommended. For the 1997-98 and 1998-99 school years there were three major strategy 
classifications: classroom-based, school-based, and community-based. The categories for 1996- 
97 were different and are not reported in this document. At the end of the year, SST coordinators 
were asked to code which of the strategies were recommended and the level of effectiveness 
(Good, Fair, or Poor). SST chairpersons were also asked to code the length of time the strategy 
was implemented (Not Used, Less than 3 weeks, 3-9 weeks, 10-18 weeks. Over 18 weeks, or Not 
Applicable). Figure 6 depicts the percentage of students in each of the three categories for the 
last two school years. 



Figure 6 

Proportions of Students in Each of Three Major Strategy Categories in 1997-98 & 1998-99. 




Classroom-based School-based Community-based 

Major Strategy 



In all three areas there has been a slight increase in the proportion of students who received 
recommendations for one or more of the strategies within the groups. One hundred and thirty 
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eight of the students did not have any strategies recorded on the final form. The average number 
of strategies recommended was 6.6, and 12 students received recommendations for 20 or more 
separate strategies, the highest being a recommendation for 24. The majority of students 
received recommendations for two to 1 1 separate strategies. Of the 23,535 separate strategies 
recommended in 1998-99, 62% were classroom-based, 34% were school-based, and 4% were 
community-based. An average of 4 classroom-based, 2 school-based and 0.5 community-based 
strategies were recommended. 

Middle school students had significantly more strategies recommended for use than elementary 
grade students, and elementary students had significantly more strategies recommended than 
high school students [F (3,3581) = 26.82, p < .01]. Students at alternative schools received the 
highest number of recommended strategies. 

Elementary school students were significantly more likely to have classroom-based strategies 
recommended than middle, high or alternative schools [F (3,3581) = 94.16, p <.01 ]. Middle 
school students were the next most likely, followed by alternative school students. High school 
students were the least likely to have classroom-based strategies recommended. Alternative 
school students were the most likely to have school-based strategies recommended. Middle and 
high school students were equally likely to have school-based strategies recommended, and 
elementary school students were the least likely to have school-based strategies recommended [F 
(3,3581) = 53.64, p <.01]. This same pattern was repeated for community based strategies [F 
(3,3581) = 46.51, p<.01]. 

Effectiveness of Recommended Strategies 

As one would expect, not all strategies recommended were implemented, and some students 
received strategies for which they were not recommended. For the classroom strategies 
implemented, the strategies were considered Fair or Good approximately 86% of the time. 
Approximately 84% of the classroom-based strategies were implemented in 1998-99, though 
care must be taken in assessing implementation as some students received strategies for which 
they were not recommended. For example, of the 628 students who had study skills 
recommended, 290 of these students received at least 3 weeks of support. A further 74 students 
who were not reported as being recommended to receive study skill s support also received at 
least 3 weeks. 

Classroom-based strategies that were used and rated are listed by order of effectiveness in Figure 
7. The columns under the heading of Strategies are the number of students for whom the strategy 
was recommended, and the number of students for whom the strategy was used. 
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Figure 7 

Classroom-Based Strategies Used During the 1998-99 School Year. 





Strategies 


Effectiveness Rating 


Strategy 


Recommended 


Used 


Good 


Fair 


Poor 


Active learning 


1549 


1401 


38% 


48% 


14% 


Assessment strategies 


1057 


956 


41% 


46% 


13% 


Differentiated instruction 


1685 


1457 


40% 


48% 


12% 


Modified environment 


1458 


1259 


36% 


51% 


13% 


Coaching 


1125 


965 


40% 


46% 


14% 


Feedback 


1361 


1177 


39% 


46% 


15% 


Flexible grouping 


1555 


1277 


39% 


49% 


12% 


Teaching styles 


729 


662 


49% 


42% 


9% 


Parental involvement 


1688 


1380 


49% 


35% 


16% 


Study skills 


628 


346 


39% 


44% 


17% 


Behavioral interventions 


1003 


860 


33% 


44% 


23% 


Technology 


334 


289 


34% 


54% 


12% 


Other classroom 


382 


254 


41% 


41% 


18% 



Figure 8 details the effectiveness ratings for the school support services strategies recommended 
and used in the 1998-99 school year. In general, all school support services that were 
implemented were rated as Fair or Good approximately 86% of the time. Approximately 72% of 
the school-based strategies were implemented in 1998-99. As with the classroom-based 
strategies, care must be taken in using this figure as some students received strategies for which 
they were not recommended. 

School-based strategies that were used and rated are listed by order of effectiveness in Figure 8. 
The columns under the heading of Strategies are the number of students for whom the strategy 
was recommended, and the number of students for whom the strategy was used. 
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Figure 8 

School Support>Based Services Strategies Used during the 1998-99 School Year. 





Strategies 


Effectiveness Rating 


Strategy 


Recommended 


Used 


Good 


Fair 


Poor 


Individual counseling 


699 


577 


44% 


38% 


18% 


Small group counseling 


311 


249 


47% 


37% 


16% 


Education program planning 


336 


254 


51% 


42% 


8% 


Consultation on 
instructional modification 


351 


291 


52% 


42% 


6% 


ESL/Migrant education 


71 


63 


54% 


25% 


21% 


Peer mediation 


74 


79 


38% 


43% 


19% 


Reading specialist 


689 


527 


53% 


38% 


9% 


Change teacher/schedule 


217 


166 


52% 


34% 


14% 


Home visit 


205 


105 


40% 


36% 


25% 


Family/community 

assessment 


109 


77 


46% 


29% 


25% 


Resiliency assessment 


22 


25 


58% 


42% 


0% 


Parent education 


231 


198 


48% 


33% 


19% 


School conference 


1314 


835 


50% 


34% 


16% 


Family counseling 


151 


113 


46% 


29% 


25% 


Consult related services 


266 


179 


54% 


34% 


13% 


Volunteer mentor 


241 


179 


51% 


34% 


16% 


WCPSS alternative school 


53 


35 


54% 


21% 


25% 


After hours school 


143 


134 


33% 


40% 


28% 


Tutoring 


801 


610 


42% 


45% 


13% 


Magnet elective 


138 


102 


46% 


44% 


10% 


After school programs 


73 


60 


47% 


39% 


15% 


Communities in schools 


70 


88 


51% 


34% 


15% 


lEP team 


731 


448 


74% 


21% 


5% 


Other school-based 


763 


396 


49% 


38% 


13% 



Community-based services were the third category of recommended strategies for the 1998-99 
school year. With the exception of legal services referrals, most community-based strategies 
were rated Good or Fair approximately 78% of the time. Approximately 63% of the community- 
based strategies were implemented in 1998-99. 

Community-based strategies that were used and rated are listed by order of effectiveness in 
Figure 9. The columns under the heading of Strategies are the number of students for whom the 
strategy was recommended, and the number of students for whom the strategy was used. 
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Figure 9 

Community-Based Services Strategies Used during the 1998-99 School Year. 





Strategies 


Effectiveness Rating 


Strategy 


Recommended 


Used 


Good 


Fair 


Poor 


Alternative school outside 
WCPSS 


69 


59 


55% 


26% 


19% 


Medical services 


316 


179 


52% 


25% 


23% 


Mental health 


220 


135 


49% 


27% 


24% 


Social services 


84 


52 


35% 


37% 


28% 


Legal services 


18 


24 


57% 


36% 


7% 


Community resource 


141 


84 


51% 


25% 


24% 


Other community-based 


11 


8 


- 100% 


0% - 


0% 



Improvement for Presenting Concerns 

For each initial presenting concern, the SST member completing the form was to indicate 
whether the broad area of concern (academic patterns, behavior patterns, 
psychological/emotional behaviors, family characteristics, and other) had improved a lot, 
somewhat, or not improved. Figure 10 summarizes the data for improvement of presenting 
concerns for the 1998-99 school year. 

Figure 10 

Improvement for Presenting Concerns in 1998-99. 
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Academic and behavioral concerns show the higher ratings for improvement. Family concerns 
are less likely to be rated as improved, even somewhat improved. 

Females were significantly more likely to have been rated for improvement for behavioral and 
psychological / emotional concerns than males. All other concerns show no significant difference 
in ratings for improvement based on sex. Black, Hispanic and Multi-racial students were 
significantly more likely to have been rated as less improved than White students in all five areas 
of concern. 

The next three figures looks at the improvement for each of the five presenting concerns for each 
of the three school types, kindergarten to five (elementary), six to eight (middle school), and high 
schools. Figure 1 1 summarizes the elementary data for improvement of presenting concerns for 
the 1998-99 school year in with percentages rounded. 

Figure 11 

Improvement of Presenting Concerns for Elementary Schools in 1998-99. 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


19% 


53% 


71% 


29% 


Behavior 


19% 


51% 


69% 


31% 


Psychological 


15% 


46% 


69% 


39% 


Family 


15% 


39% 


54% 


46% 


Other 


14% 


47% 


61% 


39% 



Ratings of improvement for elementary school children ranged from 54% for family concerns, to 
71% for academic concerns. The same categories of presenting concerns used for elementary 
schools were also used for middle schools. There was improvement for each area of concern. 
Figure 12 summarizes the data for improvement of presenting concerns for the 1998-99 school 
year. 



Figure 12 

Improvement of Presenting Concerns for Middle Schools in 1998-99. 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


17% 


44% 


60% 


40% 


Behavior 


17% 


46% 


63% 


37% 


Psychological 


18% 


43% 


61% 


39% 


Family 


13% 


42% 


65% 


35% 


Other 


20% 


42% 


62% 


38% 



Improved and somewhat improved ratings for middle school students ranged from 60% for 
academic concerns, to 65% for family concerns. In high schools, from the time of initial concern 
to the end-of-the year, all concerns improved some. Improvement for presenting concerns ranged 
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from approximately 75% (family concerns) to 83% (behavioral concerns). Figure 13 summarizes 
the data for improvement of presenting concerns for the 1998-99 school year for high schools. 



Figure 13 

Improvement of Presenting Concerns for High Schools in 1998-99. 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


32% 


41% 


73% 


27% 


Behavior 


36% 


47% 


83% 


17% 


Psychological 


39% 


43% 


82% 


18% 


Family 


32% 


43% 


75% 


25% 


Other 


26% 


51% 


77% 


23% 



The last series of data examines the ratings for improvement of the alternative schools. Students 
in these schools could be from grades K to 12. Figure 14 summarizes the data for improvement 
of presenting concerns for the 1998-99 school year for high schools. 

Figure 14 

Improvement of Presenting Concerns for Alternative Schools in 1998-99. 



Concern 


Yes, a lot 


Yes, somewhat 


Total Improved 


No 


Academic 


28% 


33% 


61% 


39% 


Behavior 


20% 


15% 


35% 


65% 


Psychological 


14% 


18% 


32% 


68% 


Family 


4% 


25% 


29% 


71% 


Other 


15% 


8% 


23% 


77% 



Improved and somewhat improved ratings were from 23% for other concerns, to 61% for 
academic concerns. The small numbers of students in these schools does make any detailed 
assessment difficult. For example, only 18 students received ratings for academic concerns, and 
the other concerns generally had 30 to 40 students rated for level of improvement. 
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EVALUATION QUESTIONS 



The purpose of this evaluation was to respond to several questions related to the types of student 
referrals, the types of strategies recommended and implemented, and the effectiveness of the 
SST program. 

Were presenting concerns more prevalent in any particular area(s)? 

As with the previous years of the program, acade mi c concerns followed by behavior were the 
most prevalent during the 1998-99 school year. Family concerns were the least frequent concerns 
noted. Over 73% of the students referred to SSTs were in elementary schools. 

A one-way analysis of variance (ANOVA) indicated that middle and alternative school students 
had significantly higher numbers of presenting concerns than either elementary or high school 
students, and elementary students had significantly more presenting concerns than high school 
students [F (3,3581) = 19.21, p < .01]. Middle and alternative school students also had a 
significantly higher number of strategies recommended than elementary students, and elementary 
students had a higher number of strategies recommended than high school students [F (3,3581) = 

26.82, p < .01]. 

Did students who were referred for behavior show improvement after SST interventions? 

Yes. The majority of students with behavior concerns showed some improvement. Students at 
alternative schools were reported to be the least improved. High schools reported the highest 
proportion of the students showing improvement in their behavioral concerns. Elementary 
students showed the next highest improvement, though the proportion of improvement was not 
too dissimilar from middle schools. Figure 15 depicts the total percentage of improvement for 
each of elementary middle, high and alternative scWls. 

Figure 15 

Improvement of Behavior Presenting Concerns in 1998-99. 



School Type 


Total Improved 


No Improvement 


Elementary 


69% 


31% 


Middle 


63% 


37% 


High 


83% 


17% 


Alternative 


35% 


65% 



Did students who were referred for psychological / emotional concerns show improvement 
after SST interventions? 

Most students were rated as having shown improvement. Students at alternative schools were 
more likely not to have been rated as having shown improvement in their 
psychological/emotional concerns. High schools showed the highest proportion of students 
showing improvement in their psychological / emotional concerns. Elementary and middle 
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schools were similar in the ratings of improvement. Figure 16 depicts the total percentage of 
improvement for each of elementary middle, high and alternative schools. 



Figure 16 

Improvement of Emotional Presenting Concerns in 1998-99. 



School T 5 q>e 


Total Improved 


No Improvement 


Elementary 


61% 


39% 


Middle 


62% 


38% 


High 


82% 


18% 


Alternative 


32% 


68% 



Did students who were referred for “other" concerns show improvement after SST 
interventions? 

Yes, there was some improvement shown for all school types during the 1998-99 school year. 
For middle and high schools, the total improvement and no improvement were similar. For 
elementary schools, total improvement was significantly lower. Alternative schools once again 
showed the least improvement. It should be noted that the “other” category is quite varied. For 
example, it includes an attendance category and an “other” category. Figure 17 depicts the total 
percentage of improvement for each of elementary middle, high and alternative schools. 

Figure 17 

Improvement of Other Presenting Concerns in 1998-99. 



School T)q)e 


Total Improved 


No Improvement 


Elementary 


61% 


39% 


Middle 


74% 


26% 


High 


78% 


22% 


Alternative 


23% 


77% 



Were strategies implemented when recommended for use? 

Strategies that were reconunended for use by SSTs were mostly implemented. Classroom-based 
strategies were more likely to be used when reconunended. Home visits and study skills had a 
higher percentage of cases where the strategies were not used despite SST recommendations. 
There are, however, a number of cases where strategies were used with students that were not 
reconunended by the SST. It is not possible to conclude from the data whether this is an 
indication of numerous visits to the SSTs or other reasons. 
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Were some strategies more effective than others? 

Strategies for the 1998-99 school year were categorized as classroom-based (see Figure 7, page 
13), school support strategies (see Figure 8, page 14) or community-based (see Figure 9, page 
15). For each individual strategy that was recommended and implemented, the percentage that 
reported an effectiveness rating of “Good” was much higher than those rated as “Poor.” This 
might suggest that the types of strategies that were recommended and implemented were quite 
appropriate and successfiil for those students. 

Classroom-based strategies were the most commonly used form of interventions reco mm ended 
and used by SST teams, followed by school-based strategies. Of the classroom based strategies, 
behavioral interventions were rated poor most frequently (23%). Other strategies tended to be 
rated poor at around 10 to 15%. 

Among all the strategies recommend, the highest rating for effectiveness was lEP teams (73% 
rated as Good). Alternative schools, after hour’s schools, social services, home visits, and famil y 
/ community assessments were rated poor the most frequently (around 25 to 30%). 

It appears that the SST program continues to provide an invaluable service to students at-risk and 
their families. For those students referred, a large majority showed improvement for their areas 
of presenting concerns by the end of the year. In addition, recommended and implemented 
strategies were generally highly successful for classroom-based, school support services, and 
community-based services. 
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RECOMMENDATIONS 

The increasing availability of laptop computers and accessible student databases may assist in 
future program evaluations. The use of scanable forms has increased the amount of data 
available for analysis, but problems persist related to missing data, and SST chairpersons seem to 
have difficulty generating the requested information in a timely manner. Consolidation of 
reporting forms and better use of shared databases may both improve data and make possible 
investigations into additional questions such as a study of the length of time between strategy 
recommendations and implementation, and identification of possible obstacles to 
implementation. 

Quantitative data provides an invaluable resource in program evaluation. However, it does not 
always provide the full picture. A qualitative component to the intervention/prevention 
evaluation could provide invaluable information. However, qualitative methods often require 
more resources for data collection and more time for analysis than quantitative data. 

Data collected for this report suggests that the effectiveness of SSTs in alternative schooling 
programs needs to be seriously questioned. Further detailed study of SST processes in the 
alternative schools is required to explain the relatively poor effectiveness ratings for the 
strategies identified by SSTs in those schools, and their high levels of ‘No Improvement’ ratings. 
Reasons for strategy implementation, or non-implementation should also be researched. This 
may result in a better initial targeting of SST discussions about the strategies to be used for 
students with different presenting concerns. 
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